Air-Writing Recognition - Part I: Modeling and Recognition of Characters, Words, and Connecting Motions

Air-writing refers to writing of linguistic characters or words in a free space by hand or finger movements. Airwriting differs from conventional handwriting; the latter contains the pen-up-pen-down motion, while the former lacks such a delimited sequence of writing events. We address air-writing recognition problems in a pair of companion papers. In Part 1, recognition of characters or words is accomplished based on 6 degreesof-freedom hand motion data. We address air-writing on two levels: motion characters and motion words. Isolated air-writing characters can be recognized similar to motion gestures although with increased sophistication and variability. For motion word recognition in which letters are connected and superimposed in the same virtual box in space, we build statistical models for words by concatenating clustered ligature models and individual letter models. Hidden Markov model is used for airwriting modeling and recognition. We show that motion data along dimensions beyond a 2D trajectory can be beneficially discriminative for air-writing recognition. We investigate the relative effectiveness of various feature dimensions of optical and inertial tracking signals, and report the attainable recognition performance correspondingly. The proposed system achieves a word error rate of 0.8% for wordbased recognition and 1.9% for letter-based recognition. We also subjectively and objectively evaluate the effectiveness of airwriting and compare it to text input using a virtual keyboard. The words-per-minute of airwriting and virtual keyboard are 5.43 and 8.42, respectively. Keywords—handwriting recognition, air-writing, 6-DOF motion, usability study.